Collaborative Group Learning

Collaborative learning has successfully applied knowledge transfer to guide a pool of small student networks towards robust local minima. However, previous approaches typically struggle with drastically aggravated homogenization when the number students rises. In this paper, we propose Group Learning, an efficient framework that aims diversify feature representation and conduct effective regularization. Intuitively, similar human group study mechanism, induce learn exchange different parts course as collaborative groups. First, each is established by randomly routing on modular neural network, which facilitates flexible communication between due random levels sharing branching. Second, resist homogenization, first compose diverse sets exploiting inductive bias from sub-sets training data, then aggregate distill complementary imitating sub-group at time step. Overall, above mechanisms are beneficial for maximizing population further improve model generalization without sacrificing computational efficiency. Empirical evaluations both image text tasks indicate our method significantly outperforms various state-of-the-art whilst enhancing